Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Assist last used: remember STT and record proactively (before connected) #3755

Merged

Conversation

jpelgrom
Copy link
Member

@jpelgrom jpelgrom commented Aug 3, 2023

Summary

Currently, Assist needs to do a few network checks before it is ready to use. If the device or network is slow this can take a second or two, especially on a watch. You can't start speaking until these checks complete.

To improve this, for the last used pipeline this PR remembers whether or not it supports STT input, and if it does it starts recording proactively/as soon as possible to avoid missing voice input while doing network checks.

Also includes two minor fixes found while working on the PR:

  • Fix potential wrong server while sending voice data.
  • Fix voice input remaining active after getting an error response.

Screenshots

These demos are on a watch, but the same applies to the main app. A 5 second delay was added for demo purposes.

Slow network, you can see the UI shift a few pixels once a connection is established:

assist-stt-cache-success.mp4

Failed connection, recording will stop and it will show the error state as usual:

assist-stt-cache-offline.mp4

Link to pull request in Documentation repository

n/a

Any other notes

 - For the last used pipeline for Assist, remember whether or not it supports STT input, and if it does start recording proactively/as soon as possible to avoid missing voice input while doing network checks.
 - Fix potential wrong server while sending voice data.
 - Fix voice input remaining active after getting an error response.
@JBassett JBassett merged commit add1955 into home-assistant:master Aug 4, 2023
3 checks passed
@jpelgrom jpelgrom deleted the assist-remember-last-pipeline-stt branch August 4, 2023 15:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants